Games of Daze/Infomagic - Games of Daze (Summer 1995) (Disc 1 of 2).iso/x2ftp/msdos/theory/evolutio/evolutio.txt

Internet Message Format | 1995-03-24 | 23KB

From: [e-mail] (neil.a.kirby)
Newsgroups: rec.games.programmer
Subject: LONG: Intelligent Computer Play -> Paper
Date: 24 Jul 91 15:08:10 GMT

The following paper was given at the 1991 Computer Game Designers
Conference.

Copyright 1991 Neil Kirby
All rights reserved.


Intelligent Behavior Without AI:
An Evolutionary Approach

Copyright 1991 Neil Kirby

Introduction
This paper describes a way to give intelligent behavior to computer
operated objects.  It traces the evolutionary approach used in the "Auto"
program of the "Bots" family of games.  The Bots system is described to
provide a context, and the results of each step in the evolution are
discussed.  While the changes made are particular to the Auto program,
the method is applicable to other programs.  The fundamentals of the
method are:
    1.  Analyze good and bad behaviors.
    2.  Quantify parameters to new algorithms.
    3.  Apply constant evolutionary pressures by repeatedly testing.
The method has proven to be quite successful.  The Auto program has
progressed from hopelessly poor play to well above average play skills
without any formal AI methods.

The Basic Approach
The basic approach greatly resembles classical Darwinistic evolution.
Start with simplistic behavior, however bad.  Simple behavior is best
described as whatever is easiest to code.  Analyze the failures of the
simple methods.  If available, observe the differences between successful
behavior, especially that of human players, and the failures of the
computer players.  The most difficult step is to identify the parameters
that can be used to control successful behavior.  Once this has been
done, use regular code to allow the program to react to these parameters.
The process is completed by repeatedly testing in play.  The ease of fast,
repeated play afforded by computerization accelerates the process.  This
approach is easily observed in the Auto program.

The Bots Family of Games
The Auto, Aero, and Bots programs make up the Bots family of
multiplayer, tactical ground and air combat games.  These run under the
Unix System VTM operating system and are written entirely in the C
programming language.  Bots and Auto deal with futuristic ground combat
units loosely comparable to tanks while Aero deals with flying futuristic
aircraft.  The Bots and Aero programs are used by human players and
Auto is the computer managed equivalent to Bots.  The Bots family of
games supports but does not require team play.

Since the  Bots family are multiple player, multiple process games, events
are performed simultaneously.  A unit can find out the actions of other
units only after committing its own actions.  The events that make up a
turn are motion, followed by active fire, and ending with passive fire.  After
every ten turns, repair and resupply are made available to a unit.  The
manner in which a unit conducts its motion and fire are strongly
influenced by the construction of the unit.

Building a Unit
The units used in Bots and Auto are built according to the tastes of the
player running them.  A unit has six armor facings protecting its internal
systems from damage.  Armor is ablative; each point of damage reduces
the amount of armor by one.  Once a given armor facing is reduced to
zero, all remaining damage is applied to the internal systems.  There are a
number of internal systems to protect: life support, control, chassis,
reactor, batteries, hardpoints, weapons, and ammunition.  Each has a
cost and weight associated with it, forcing economic compromises to be
made to build a unit to meet a given cost.  The mobility of a unit is based
on the amount of power available and the current mass of the unit, both of
which can change in play.

Weapons
There are a variety of weapons systems available, each with different
characteristics.  Weapons that require ammunition usually need little
power while power-based weapons require much more.  The lasers are
relatively light in mass, short-range, and power hungry.  Mortars have a
minimum range and require heavy ammunition, but they also have a long
maximum range and do not require line of sight to fire.  Autocannon inflict
moderate damage to moderate ranges, but they too, require a large
amount of moderately heavy ammunition.  Miniguns are lighter, smaller,
short-range versions of the autocannon.  Missiles have varying ranges
and warheads, but they can be shot down by point defenses, and they too,
are heavy.  Particle beam weapons are highly effective in a narrowly
focused range.  Aside from mortars, missiles, and particle beams, most
weapons improve as range shortens.

The non-offensive weapons systems are stealth, missile defense, and
hardpoints.  Stealth, which is expensive,  allows a unit to approach others
and remain unseen.  Missile defense attempts to shoot down incoming
missiles.

Hardpoints are mounts for optional, single-shot items.  The most common
hardpoint load is a jump pack.  Jump packs allow a unit to jump over
obstacles such as water.  They increase the mobility of a unit greatly but
can only be used once.  The other common hardpoint load is an anti-
aircraft missile.

Play Balance (Rock, Paper, Scissors)
The selection of a weapon determines much about the unit.  Units with
long-range weapons usually require heavy ammunition.  Their great
weight means limited mobility.  Limited mobility implies an inability to
escape from units with high mobility and short-range weapons.  It is
axiomatic that short-range weapons require high mobility.  Therefore,
short-range weapons must be light to be effective.  These trade-offs are
required for play balance among a wide diversity of systems.  Mortar
units, for example, were popular on teams but few people actually wanted
to play them.

To be successful in play, a unit must maneuver effectively, fire its
weapons, and evade enemy fire.  It should distribute enemy fire across
multiple armor facings.  If team mates are present, a unit should
coordinate fire among the team members to combine fire against single
enemy units.  If a unit takes damage, it should flee until it is repaired.  The
richness of detail in the Bots family of games is comparable to that of Star
Fleet BattlesTM.

Auto units are not allowed to cheat, so the behavior code cannot access
information that a human player in the same situation would not have.  The
advantage of access to hidden data was considered at first, but has
proven to be unnecessary.  No Auto program was written for Aero units.
The success of Auto units on the ground removed all demand for an Auto
equivalent to Aero.

The Original Algorithm
The original algorithm for Autos was quite simple.  The closest enemy unit
was selected as the target.  All other units were ignored.  The Auto turned
to face the enemy and attempted to close.  (See Figure 1)  After
maneuvers were complete, the unit attempted to fire its weapons.  The fire
target was the closest enemy within the arc of the weapons fire.  The unit
would not flee if damaged.  The unit would not jump unless blocked by
water, buildings, or steep slopes.

Spreading Out Damage
There were many failures with the original algorithm.  Foremost among
these was the propensity for the unit to lose its center forward armor and
take severe internal damage while the left and right forward armor facings
were pristine.  The solution to this problem was for the unit to shorten the
amount it moved to save sufficient mobility to rotate after its motion.  (See
Figure 2)  This rotation would bring the strongest forward armor to face
the closest enemy unit.  Relative bearing and the integer values of the
forward armor were the parameters that enabled this change, which
typically tripled the survival of all units.  It especially allowed units with
short-range weapons to close to their effective ranges.  This algorithm
became known as the basic closure code.


Self Assessment
The modified algorithm still had major problems.  During a long closure, a
unit would loose all three forward armor facings, continue closing, and
take fatal internals.  The fix involved self assessment.  Before a unit
started maneuvers, it would evaluate the state of its forward armor,
internal systems, and ammunition.  If all three forward armor facings were
below minimum, half of any internal system was destroyed, or if
ammunition was gone, then the unit would declare itself to be unfit for
battle and flee.  It would turn its strongest rear armor toward the closest
enemy and move as far away as possible.  This is just a slightly modified
version of the basic closure code.  If the unit had a jump pack available, it
would jump as far away as possible.  It would choose not to shoot power
based weapons to save energy for mobility.  The parameters for this
change were simple integer numbers: armor, systems, and ammunition
counts.  This change increased the long term survivability of the Auto
units greatly.  After it, destroying a unit often required a long chase.  In
rolling terrain, enemy units would simply disappear when they were
seriously damaged.

Battle Range
At this point, Auto units were interesting but not threatening.  Their
maneuvers were highly predictable.  At point blank range a unit would
close to zero range and stop.  Since all motion in Bots is simultaneous,
human players would simply move forward and turn around, finding
themselves at near zero range facing the back of the Auto unit.  Fatal
internals for the Auto unit often soon followed.  For long-range units such
as mortars, and specific range units such as particle beam weapons,
continuos closure was especially counter-productive.  Mortar units have a
minimum range requirement, and care very little about range otherwise.
In the laser family, where closure improves damage, closure also gives
the advantage to the lightest and shortest ranged lasers.  The fix was to
establish a battle range (known as BRG) for each weapon type.  The
parameter used with BRG was range.

BRG numbers were tuned during play.  While many weapons benefit from
close range, BRG was picked to balance survivability and the ability to
inflict damage.  Long-range units would prefer to stay at range, denying
short-range units any fire.  If a unit found itself too close, it would back up
as much as it could.  Yet backing up costs twice as much as forward
motion.  BRG helped to keep the long-range support units out of trouble,
and it helped all units survive combat with a short-range unit.  Short-range
units were still too predictable and easy for human players to destroy.
BRG was the evolutionary step that set the stage for a major milestone in
Auto evolution.

The First Major Milestone Change: New Maneuver Code
The problem with the maneuver code was predictability.  The new
maneuver code dealt with the answers to two questions:  "Where can a
unit go?"  and  "Where does a unit want to go?"  The first question is
answered by computing the mobility of a unit, which is based on its power
and mass.  The unit can get to roughly any point within a circle, centered
on the unit, of radius equal to the mobility of the unit.  When computing the
radius of the mobility circle, sufficient mobility is deducted from the radius
to provide for turns.  The second question is answered by BRG.  If the
target does not move, the most effective place to be is some place on a
circle, centered on the target, of radius BRG.  (See Figure 3)  If the two
circles do not intersect, the existing basic closure code is used.  If the
circles intersect, there are three possibilities.

The first possibility is shown in Figure 4.  This is the most dangerous case
for the target, and the best case for the attacker.  This situation is most
often seen when a high mobility unit carrying short ranged weapons
attacks.  The unit will pick a random point on the BRG circle, move to it,
and turn to face the target.  For the target to evade, it must either move far
enough to get out of range, or it must move behind the random point on the
BRG circle where the attacker will appear.  Neither of these evasions is
easy.  The first requires good mobility, and the second requires good luck.
If the target does move far enough away to deny weapons fire, the
attacker will have more power available to move in the next turn.

The second possibility is shown in Figure 5.  This is a good case for any
attacker.  This situation is most often seen when a medium or high
mobility unit carrying medium-range weapons attacks.  The unit randomly
picks one of the two intersection points, moves to it, and turns to face the
target.  Only a target with more mobility than the attacker will be able to
out maneuver the attacker.  As Figure 5 moves toward Figure 4, the
attacker becomes nearly impossible to evade.

The third case is shown in Figure 6.  The mobility circle is inside the BRG
circle, which means that the attacker is too close.  This is usually the
case when a low mobility unit carrying long-range weapons is closer than
desired to  an enemy unit.  Here the unit determines if it would get farther
away by backing up, or by turning away, moving forward, and turning again
to face the target.  It chooses the way that takes it farthest away and
executes that maneuver.

BRG and computed mobility, the parameters of the change, are combined
in an effective algorithm.  The effect of the first milestone change was
dramatic.  Attacking units were much more effective (fleeing units were
unaffected).  Human players could no longer destroy Auto units without
pause.  Close combat maneuvers became very dicey.  Novice players had
less than a 50% chance of winning a one-on-one duel.  In single combat,
Auto units played a solid, if uninspired, game and they never made stupid
errors.

Initial Team Play: Communications
The team play of Autos still lacked a great deal - they fought as a mob,
rather than as a team.  Any unit that could see no targets would pick a
random patrol point on the map and head toward it.  This scattering
behavior was good for finding hidden enemy units, but poor for fighting
enemy teams.  If one unit on a team was engaged, the rest of the team
would ignore the combat unless they too could see the target.  Evolution
continued, as Auto units learned basic communications skills.  A unit that
could see no targets would ask its team for targeting information.  The
other units (Aero, Auto, and Bots) on the same team would reply with the
map coordinates of the targets that they could see.  These coordinates
could be used the next turn to plot motion.  Units that did not require line-
of-sight to fire, such as mortars, would fire blindly on the map coordinates
given.  The effects of this change were two-fold.  It allowed high mobility
units carrying short-ranged weapons to close upon enemy units that they
could not see.  This meant that once an enemy unit was spotted, all
unoccupied Autos would head toward it, even in rolling terrain.  The
second effect provided mortar teams with something productive to do.
Communicating map coordinates proved to be an effective change.  To
the enemy units, it meant to expect mortar fire whenever spotted by any
member of the Auto team.  A particularly nasty combination was to be
stalked by an unseen stealth unit that was broadcasting coordinates to its
mortar teammates.  This change tended to keep mobs of Autos together,
but they still fought as a mob.

The Second Major Milestone Change: Improved Team Play
Targeting was still based on the closest enemy that could be fired at.  In
team play, a lone unit could distract part of a team of Autos and play Pied
Piper while the rest of the Piper's team destroyed the other part of the
Auto team.  The Piper might not survive, but while it was being destroyed,
multiple enemy units would be destroyed, tipping the battle against the
Autos.  The second milestone change increased team play.  Team play
was based on the concept of the team target.  The enemy unit that
appeared to be hurt the most would be designated as team target.  This
designation was based on the volume of fire an enemy had absorbed.  The
team target would always be the preferred target for motion control.  If the
team target was between the minimum and maximum effective range for
the weapons fired, it would be the preferred target for fire as well.  The
results were often devastating and occasionally quite strange.  The
devastating part stemmed from the fact that a single unit, often already
damaged, could now take all the mortar, long-range missile, autocannon,
and much of the short-range laser fire that an entire team could inflict.
The strange part would happen when an Auto unit would move directly
past and ignore a closer target in order to help destroy the team target.
But if the team target were not a viable target, each unit would fire as
before, usually at the closest target.

Mishaps in team play were now fatal.  Teams of experts were now doing
well to achieve a kill ratio of 2.5:1 in favor of the experts.  Novice teams
required two to five times numerical superiority to win.  The parameters
used had been developed in earlier stages of evolution.

Attack Jumps
Evolution continued as details were refined to solve minor problems.  One
problem was that Autos would only use a jump pack to flee or to cross
water.  This meant that they would often have an unused jump pack at
reload time, when a new jump pack would be available.  The fix was
simple; units still carrying a jump pack that were near their reload time
would be free to jump to increase their mobility.  The parameter used was
the unit's turn counter, which already existed.  This change made it
dangerous to rely on predictions about the mobility of an Auto unit.  A fast
moving unit that had been moving four ranges per turn would suddenly
jump eight and move four more for a total of twelve ranges of motion when
only four were expected.

Dealing With Aeros
The final changes involved dealing with Aeros.  Bots and Autos usually
move one to four ranges per turn.  Aeros need to move at least ten to
retain airspeed, and speeds of twenty to fifty are normal.  Normal altitudes
for Aeros typically run from eight to twenty ranges of altitude.  The first
problem Aeros created was that Autos would attempt to set up motion
based on an Aero target.  Since the motion algorithms are optimized for
non-moving targets, using an Aero to set up motion was ineffective for
units with a short BRG.  It was ineffective for units with a long BRG if the
Aero was close by, since the bearing of the Aero would change
dramatically.  Even if perfect predictive algorithms were available, only
units with a long BRG have sufficient range to hurt Aeros at altitude.  The
important changes were range based.  All units that had a BRG of 25 or
higher would prefer any visible enemy Aero to all other targets for fire and
motion.  Units with lower BRG would ignore Aeros when plotting motion,
but if an Aero was somehow in acceptable range, it would be the preferred
fire target.  At reload time, Autos remembered if they had seen any Aeros
since last reload and changed their hardpoint weapons mix to prefer anti-
aircraft missiles.  The changes decreased the survivability of Aeros
greatly.  Aero targets were preferred even to team targets, often with the
same deadly results.  The much-maligned mortar units provided greatly
needed flak fire.  Other less popular long BRG units gained a new role.
Also, the advent of the stealth anti-aircraft missile battery eased the
problems ground units had with Aeros.

Future Evolution
There are still several deficiencies with the Auto program.  Complex
terrain tends to mystify Autos, especially urban terrain and bridges.  An
Auto unit will occasionally present weak or down armor toward one enemy
while fighting another.  This single mindedness also shows up when a
damaged unit flees.  Motion directly away from the closest enemy, which
is always correct in one-on-one, can be fatal when intermixed with an
enemy team.

Observations About The Process
Many observations are worth examining.  The first of these is that the
process of analysis and synthesis is regenerative.  The initial BRG
change set the stage for the first milestone change (maneuver code).
BRG also led to min/max range, which when coupled with
communications led to the second milestone change (team play).  Some
of the other analysis that went on made it possible and more probable that
algorithms could be generated to improve poor behavior.

Not as obvious in the paper is the fact that intensive testing helps drive
the process.  The entire evolutionary cycle mentioned above took place at
lunchtimes over the course of a year.  The testing allowed the author to
observe human behavior in order to model it, and it clearly showed any
deficiencies in the algorithms.

One very hopeful point is that simple methods often suffice.  The long
closure code and the code for flight is simple but has proven to be very
effective.  The early code, which had numerous deficiencies, was still
interesting and fun.  Between the first and second milestone changes,
there was, occasionally, the semblance of team behavior by default.  If a
given enemy unit was the best or only target available to a team, the entire
team fired upon it.  In rolling terrain this would be fatal as one unit
encountered groups of enemy units.

A final point is that the code is well behaved.  It is small, runs rapidly, and
is easy to follow.  The behavior control parts of the code do not require
any extensive calculations.  The code is short, with only two files.  In fact,
the Auto program has less source and a smaller executable than the Bots
program.  The increase in size from automation is more than offset by the
reduction in size gained by deleting the user interface.  The control flow of
the code is relatively easy to follow, and therefore easy to modify.


Conclusions:
The evolutionary method is a viable alternative to exploring AI
methodologies.  The method is universally available to programmers who
have sufficient time for repeated testing.  It is well suited to game
programs in which the computer and human players deal with the same
objects and information - - the programmer can model computer behavior
on successful human behavior.  The method fails when the programmer
cannot identify the keys to changing poor behavior and the default simple
methods are ineffective.  For the multiplayer, tactical combat games that
this author has written, the methods have produced effective code that is
fast, compact, and often pleasingly simple.

    TM UNIX is a trademark of AT&T Bell Laboratories.
    TM Star Fleet Battles is a trademark of Amarillo Design Burea.